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Creative Collaboration Between Chevron and CSUB: 

Research Experience Vitalizing Science - University Program 

Abstract 

Since 2007, Chevron has funded the Research Experience Vitalizing Science - University 
Program (REVS-UP), which lasts four weeks each summer to develop Science, Technology, 
Engineering, and Mathematics (STEM) projects at CSUB. Over the past six years, a total of 26 
STEM professors have led the completion of 1 13 research projects with the support of 90 CSUB 
student assistants, 384 high school students, and 84 teachers from K-12 schools. The hands-on 
research experience has rejuvenated student interest in STEM discoveries and enhanced subject 
competency for K-12 teachers. In addition, professors received the team support to fulfill their 
research agendas, and CSUB student assistants gained service-learning experiences. This report 
concludes with five recommendations to sustain the trend of program enhancement. 
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Creative Collaboration Between Chevron and CSUB: 

Research Experience Vitalizing Science - University Program 

California State University, Bakersfield (CSUB) and Chevron Corporation have 
established a partnership since 2007 to offer Research Experience Vitalizing Science - 
University Program (REVS -UP) for high school students in Kem County. Spreading across a 
region as large as the state of New Jersey, Kem County relies on CSUB as the only state 
university to support public higher education in its culturally diversified mountain, desert , and 
valley communities. REVS-UP has been designed to use the university capacity to expand 
research opportunities for high school students. The program lasts four weeks each summer to 
develop Science, Technology, Engineering, and Mathematics (STEM) projects led by CSUB 
professors. Meanwhile, K-12 teachers are invited to enhance professional development in STEM 
education. The research team also includes CSUB student assistants to strengthen service- 
learning experiences and act as role models for high school students. The research-based 
collaboration not only creates a partnership beyond K-12 education, but also conforms to the 
virtue of STEM inquiries advocated by national standards of professional organizations 
(National Council of Teachers of Mathematics, 2000; National Research Council, 2011a). 

Innovative Approach to STEM Education 

Although mediocre performance of American students has been reported at K-12 levels 
from international testing (see Martin, Mullis, Foy, & Stanco, 2012; Mullis, Martin, Foy, & 
Arora, 2012), the “crisis” in STEM education does not seem to impact the vitality of U.S. 
economy in global market competition (Berliner, 2013). In part, this is because “high school 
students are encouraged to dig into the rich mine of knowledge from college curricula” (Wang, 
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2013, p. 13). Concurrent college enrollment of high school students has allowed U.S. students to 
have an early start in higher education (Berliner & Biddle, 1995). In contrast, high performing 
countries in East Asia confine student learning within the boundary of topics covered by college 
entrance examinations. Therefore, international reports of student performance have been 
downplayed by education researchers due to their inability to reflect the enrichment of STEM 
education beyond K-12 settings (Ravitch, 2013; Rotberg, 1991). 

REVS-UP is built on the strength of concurrent enrollment to engage high school 
students in inquiry-based learning. As suggested by Julio Blanco, CSUB Dean of Natural 
Science and Mathematics, “The way you leam science is not by reading about science but by 
doing science” 111 . Research projects developed from REVS-UP are grounded on mathematical 
and scientific explorations. When the U.S. economy entered a recession in 2007, politicians 
attempted to use standardized test scores to justify relentless budget cuts on education. This 
“quick fix” has skewed curriculum alignment toward specific tests, and thus, deprived student 
interest in open-ended STEM explorations. As Sadler, Sonnert, Hazari, and Tai (2012) observed, 

Preparing students for careers in science, technology, engineering, and mathematics 
(STEM) is at the forefront of K-12 educational concerns in the United States. ... Adding 
urgency to this concern is the fact that some STEM fields have suffered from declining 
student interest, (p. 412) 

To address this issue, REVS-UP has taken an innovative approach to rejuvenate student interest 
in STEM discoveries. Through the team effort, REVS-UP has supported CSUB professors to 
fulfill their research agenda and created a new community of learners each summer. In 
particular, high school students earned university credits for working on REVS-UP projects, 

[1] http://www.tumto23.com/news/your-neighborhood/southwest-county/csub-receives-500k- 
from-chevron-for-stem-programs 
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CSUB student assistants gained service-learning experiences, and K-12 teachers developed 
STEM subject competency to strengthen education quality for the general public. 

Revitalizing STEM education is crucial to maintaining U.S. preeminence in the 21st 
century (President’s Council of Advisors on Science and Technology, 2012). It has been 
estimated that one million more college graduates are needed across the nation in the next 
decade. “At current rates, American colleges and universities will graduate about three million 
STEM majors over the next decade, so an increase of one million would require a whopping 33- 
percent increase” (Gates & Mirkin, 2012, p. 1). The key national interest has been addressed in 
REVS-UP by attracting local students to STEM disciplines before their entrance into college. In 
addition, state figures showed that only 12.1% of Kern County high school juniors and seniors 
took Advanced Placement (AP) exam(s) in 2006-07, far below the state average of 22.4% [2] . 
Therefore, this program is much-needed for college-bound students to expand their advanced 
learning opportunities in this region. 

Besides preparing STEM majors, “Personal and societal decisions in the 21st century 
increasingly require scientific and technological understanding. ... Targeting all students, not 
just those who will pursue postsecondary education or careers in STEM or STEM-related fields, 
will better prepare citizens to face the challenges of a science- and technology-driven society” 
(National Research Council, 201 lb, p. 5). Through a balanced program design, REVS-UP has 
demonstrated its creativity in meeting the dual needs of STEM education - The inclusion of K- 
12 teachers directly facilitates compulsory education for local citizens, and the internship 
opportunities are offered to CSUB student assistants to benefit those who have already declared a 
major in STEM fields. 


[2] see 2012 annual REVS-UP report, p. x. 
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Since local students are more likely to seek employment in the same region, community 
leaders recognize the vital importance of supporting STEM education at CSUB. In particular, 
Chevron has been headquartered in California for 130 years. Over the past six years, the 
company leadership repeatedly confirmed its commitment to supporting STEM education. One 
Vice President acknowledged that “Chevron depends on a workforce skilled in math and 
science” l3J , and another Vice President echoed that “It's absolutely critical for companies like 
Chevron to have a steady pipeline of young engineers and scientists in all the places we do 
business so we are always looking for effective ways to stimulate that interest” 141 . CSUB 
embraced the same dedication to STEM education, and endorsed a strategic initiative to extend 
excellence in mathematics and science 151 . The shared vision has forged a private and public 
coalition between Chevron and CSUB to sustain funding of REVS-UP over a seven-year period 
during 2007-2013. 

Research Questions 

While new STEM projects are being developed in 2013, past research outcomes have 
been tracked online, and the trend data show completion of 1 13 STEM research presentations 
during 2007-2012 (http://www.csub.edu/stem/). Built on the ongoing program documentation, 
this report focuses on summative evaluation of REVS-UP since 2007. Three cohorts of research 
questions have been developed for this investigation: 

I. Concurrent Enrollment of High School Students 

1. Did REVS-UP attract qualified high school students to STEM fields? 

2. Were those students highly motivated during project collaboration? 

[3] http://www.calstate.edu/newsline/2007/n20070223bakl.shtml 

[4] http://www.bakersfieldvoice.com/content/research-abounds-summer-through-fifth-annual- 
revs-program 

[5 ] http ://w w w .c sub .edu/stem/Report2007 .pdf 
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3. How successful was REVS -UP on college preparation? 

II. Systematic Support from Other REVS-UP Participants 

4. In what aspects did REVS-UP strengthen professional development of K-12 teachers? 

5. How effective did CSUB student assistants perform as REVS-UP partners? 

6. How well did professors demonstrate academic leadership in STEM research? 

III. Sustainable Trends in Capacity Building 

7. In what areas did REVS-UP show continuous improvement over past six years? 

8. What progress has been made to sustain REVS-UP impact in K-12 settings? 

9. What recommendations can be made to extend success of REVS-UP beyond this 
summative evaluation? 

These questions are designed to comprehensively address the needs of this summative 
evaluation. According to Maduas, Stufflebeam, and Scriven (2000), "Ultimately the value of 
program evaluation must be judged in terms of its actual and potential contributions to improving 
learning, teaching, and administration" (p. 18). The learning aspect has been examined by three 
questions in Cohort I. The teaching component is supported by evaluation of systematic support 
from other REVS-UP participants (Cohort II). The third cohort of questions is developed to 
facilitate improvement of the program administration. Altogether this evaluation report focuses 
on what works, for whom, and in what context in order to sustain the ongoing trend of program 
enhancement. 

Methodology 

To address these wide-ranging questions, qualitative and quantitative data have been 
gathered from multiple sources, including past annual reports, student transcripts, professor 
vitae, program records, as well as surveys and interviews of REVS-UP participants. Whenever 
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possible, data accuracy has been verified through result triangulation. In particular, the 
cumulative number of REVS-UP participants has been validated according to project assignment 
forms, STEM course enrollments, and past program reports. While the enrollment figures 
provide baseline information, program completer counts are confirmed by STEM project 
presentations in annual reports. Likewise, high school GPA information has been corroborated 
across applicant databases, phone interviews, and school counselor feedback. College admission 
results are based on high school exit surveys, student questionnaires, and e-mail communications 
with school principals and parents. 

The extensive effort is built on the collaborative support of key stakeholders. REVS-UP 
Director, Professor Andreas Gebauer of Chemistry, remains at the forefront of technology 
advancement, and incorporates an electronic dropbox to enhance the security of document 
collection. He also developed a useful query link on the iStrategy platform to track aggregation 
of student demographic data (gender and ethnicity) across the past six years. The program 
coordinator, Ms. Andrea Medina, ran the daily operation of REVS-UP, and established a 
database to ensure timely distribution of teacher and student questionnaires. She gathered vitae 
files from K-12 teachers and CSUB professors, and played a pivotal role in monitoring the data 
collection. In addition, two REVS-UP alumni extended their assistance to contact past 
participants through various channels, including personal e-mails, Facebook, and Linkedln. 
CSUB’s Institutional Research Office (IRO) and Computer User Support Division also created 
an iStrategy account for record gathering from the university PeopleSoft system. To figure out 
the eventual college-going rate, IRO also submitted a search task to the StudentTr acker system to 
identify past REVS-UP participants from National Student Clearing House. 
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Those efforts played an important role in overcoming several barriers to data gathering. 
First, academic quality of high school students could be represented by GPA. But not all schools 
were willing to provide the information, and one principal indicated that “I am not sure if we are 
allowed to release the information without consent.” Instead of collecting the information from 
the high school site, Question 1 of this report is addressed through an analysis of REVS-UP 
applicant information from the CSUB side. While the application itself might serve as an 
indicator of student motivation, more explicit measures have been incorporated in survey 
questionnaires to gather the pertinent information for Question 2. Because a good portion of 
REVS-UP participants have yet to go to college, data filtering has been conducted for those past 
students eligible for college admission, and an intensive effort has been made to reduce the rate 
of missing data at the level of past high school students. 

Recent changes also demand additional attention on the data tracking. Living in high- 
needs school districts, some teachers and CSUB student assistants moved to other locations 
and/or took different jobs during the economic recession. Meanwhile, CSUB endured constant 
budget cuts since 2007, and was forced to reduce its staff. The REVS-UP coordinator assumed 
the current position in 2010, and has been responsible for multiple projects since then. After the 
initial questionnaire distribution, the coordinator followed up with two rounds of e-mails to 
collect responses pertinent to Questions 4 and 5 of this report. Similarly, quantitative data from 
project evaluation are incorporated with document analyses of faculty vitae to demonstrate 
academic leadership of CSUB professors in STEM inquiries (Question 6). 

According to Tom Angelo (1999), former director of the national assessment forum, 
“Though accountability matters, learning still matters most” ( ( J[. 1). To sustain the trend of 
capacity building, a relatively new research method named “social network analysis” has been 
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employed to disentangle variable relationships for continuous improvement (Question 7). 
Evaluation of Question 8 is focused on the sustainable impact of REVS-UP outcomes, such as 
the development of STEM lab manuals and curriculum materials, in K-12 settings. This report 
concludes with future recommendations for improvement of REVS-UP effectiveness beyond the 
existing accomplishments (Question 9). 

Stufflebeam and Webster (1984) pointed out, “Question-oriented studies are so labeled 
because they start with a particular question and then move to the methodology appropriate for 
answering that question” (p. 27). In this report, research methods have been designed to conform 
to professional standards. For more than three decades, the Joint Committee on Standards for 
Educational Evaluation advocated four essential attributes that characterize good evaluation 
projects, utility, feasibility, propriety, and accuracy (Yarbrough, Shulha, Hopson, & Caruthers, 
2010). In this REVS-UP evaluation report, the utility standard has been addressed by 
development of research questions for improving learning, teaching, and program administration. 
The feasibility standard is justified by credibility of the evaluation procedure in a realistic setting. 
In addition, consent requirement and data security have been considered in human right 
protection according to the propriety standard. Finally, result triangulations have been 
incorporated to enhance alignment with the accuracy standard. 

Findings 

In describing “models, metaphors, and definitions in evaluation”, Madaus and Kellaghan 
(2000) pointed out, 

Which model then should we choose for the evaluation of our education projects? The 
emphatic answer is, “None of the above; that’s the wrong question to ask.” Rather than 
starting from a pet approach, we should begin with a consideration of the evaluation 
questions that could be addressed, the issues that must be addressed, and the available 
resources, (p. 25) 
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In this report, nine questions were derived to address REVS-UP outcomes pertinent to learning, 
teaching, and program administration. Key issues have been examined in the methodology 
section, and available resources have been employed to support multilevel data collection. 

“The purposes of outcome evaluation/value-added assessment systems are to provide 
direction for policymaking, accountability to constituents, and feedback for improving program 
and services” (Stufflebeam, 2000, p. 45). At the core of the REVS-UP initiative is the 
concurrent enrollment of high school students in a college setting. While high school records set 
a baseline, college performance represents outcome measures for value-added assessment. To 
facilitate the result tracking, quantitative and qualitative findings have been organized according 
to the original sequence of research questions. 

Ql. Did REVS-UP attract qualified students to STEM fields? 

Although most participants in 201 1 and 2012 are still in high school, Figure 1 shows a 
steady increase of REVS-UP application each year. As the program entry becomes more 
competitive, REVS-UP has developed a reputation as a “must do” program for high school 
students in Kern County. 

Figure 1: Trend of Increase in REVS-UP Applications 

2010 2009 



Along with the expansion of applicant pool is a growing demand to select qualified 
students from local schools. French, Homer, and Robins (2010) maintained that “high school 
GPA is a positive and statistically significant predictor of educational attainment and earnings in 


14 


adulthood” (p. 2). Figure 2 indicates that REVS-UP participants have been maintaining an 
average GPA above 3.5 over past six years. Thus, the increase of student applications has not 
compromised REVS-UP’s original purpose of attracting qualified students. During 2007-12, 384 
high school students participated in REVS-UP. Excluding the portion of students currently 
enrolled in high school, 257 students have reached a point beyond high school graduation. 

Figure 2: Trend of Applicant GPA During 2007-2012 



Year 


While high school records are confirmed by information from application forms, a study 
of college preparation hinges on the articulation of GPA results in higher education. In this 
regard, REVS-UP acts as a Summer Bridge program between secondary and tertiary education. 
As one of the K-12 teachers acknowledged, “The most rewarding aspect of REVS-UP is to see 
high school students opening up their view of science and math on the college level.” Despite 
the difficulty of student tracking, 179 of the past REVS-UP participants are confirmed in college. 
Since the remaining participants are unreachable through the past records, a conservative 
consideration is to leave them in a non-college bound category. Even with this conservative 
estimate, the college-going rate for REVS-UP participants has reached 70%, more than doubling 
of the state-reported 33.2% college-going rate for Kem County 161 . 


[6] http://www.cpec.ca.gov/StudentData/CACGRCounty.asp). 


15 


Although REVS-UP was primarily designed to attract students to STEM fields, non- 
STEM majors may also benefit from the enhancement of scientific literacy and/or mathematical 
proficiency to face the challenges of a science- and technology-driven society. Figure 3 shows 
that 102 out of the 179 college-bound students have chosen a STEM major, and 29 participants 
indicated a non- STEM major. 

Figure 3: Partition of REVS-UP Respondents By College Majors 



Interview results further indicate the impact of REVS-UP on student career choice. One 
past student recollected, 

Before attending this program, I was indecisive about my future, and I was about to enter 
my junior year in high school. Since this program, I have graduated early, tested into 
CSUB, and I am now on my way to being a natural products chemist and a 
pharmaceutical rep. 

Another prospective student of CSUB concurred, 

What I experienced in REVS UP has encouraged me to continue pursuing an education in 
engineering after graduation from Bakersfield High School in 2014. CSUB has been one 
of my first college choices and it would be wonderful to be part of the School of 
Engineering. I find the working environment of college classes much more rewarding 
than high school and I look forward to encountering it again. 

In summary, “Correlational evidence suggests that high school GPA is better than 
admission test scores in predicting first-year college GPA” (Sawyer, 2013, p. 89). REVS-UP has 
been maintaining an average high school GPA above 3.5 over past six years. In addition, past 
participants have demonstrated a much-higher college-going rate in contrast to the state average. 
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Finally, interview findings further reconfirmed the positive impact of REVS-UP on attracting the 
majority of participants to STEM fields. 

Q2. Were high school students motivated during the project collaboration? 

Motivation is a psychological construct that compels and/or reinforces an action toward a 

desired goal (Marsh, Hau, & Kong, 2002). In REVS-UP, the goal could be extended to future 

career orientation beyond the teamwork on a STEM project. More specifically, one high school 

student expressed an intrinsic motivation to join the Chevron workforce: 

One of the reasons why I am staying here is because I know Bakersfield offers 
engineering internships. Not only that but, Chevron is amazing due to the fact of what 
they given over the past few years. I would like to work in Chevron one day if I had a 
chance. 

Although educators found a consistent decrease in intrinsic motivation of STEM subjects 

during grade 3-9 (see Harter, 1981), the result from REVS-UP has reversed the chronological 

trend by rejuvenating student interest toward STEM investigations. One student recounted, 

I applied and got accepted to CSU Bakersfield’s STEM 2010 summer program. This 
program further influenced my interest in science because I was given the opportunity to 
participate in the chemistry segment of the program. I learned a lot of new vocabulary, 
and laboratory and analytical skills. 

Besides those interview responses to the external evaluator, more information has been 
gathered from internal REVS-UP team members. Nineteen K-12 teachers provided survey 
responses, and 14 teachers reported having motivated high school students in their team (Figure 
4). The result reconfirmed development of intrinsic motivation during the research process. One 
teacher indicated her pleasure of “Seeing the Tight bulb’ moment on the students faces”, and 
another teacher noted that “observing students experience scientific research was wonderful.” 

In comparison to intrinsic motivation, Smith (2004) asserted that “A student's response to 
extrinsic motivators is a very individual thing. What works for one student won't work for 
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another” ( C J[. 5). One extrinsic motivator was the additional college credit students would earn for 
REVS-UP participation. A few students indicated that they were more interested in STEM 
learning, and neglected the college credit they could have claimed. Therefore, intrinsic 
motivation appeared to have played a dominant role in supporting STEM participation of high 
school students. 

Figure 4: Teachers’ Report of High School Student Motivation 
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It should be noted that intrinsic motivation does not always emerge as visible excitement. 
Figure 5 indicates that apparently “not- so-motivated” students may still serve as good or very 
good partners. Hence, it is the mutual reinforcement between motivation and performance that 
provides a sustainable support for STEM learning in REVS-UP. 


Figure 5: Relationships Between Student Motivation and Partnership Performance 


very good partner 
good partner 
acceptable partner 
insignificant partner 



FREQ. 

10 

7 

1 
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I I not motivated I I motivated 


While stronger motivation leads to better performance, the relationship could be 


reciprocal as well (Marsh, Hau, & Kong, 2002; Wang, Oliver, & Staver, 2008). As Confucius 
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pointed out, “The more a man learns, the more he knows his ignorance” 171 . Sensing the strong 
desire to learn, around 90% of the teacher respondents rated high school students as very good or 
good partners (Figure 6). 


Figure 6: Teachers’ Rating of Student Partnership 


very good partner 
9 


insignificant partner 
1 


acceptable partner 


good partner 
7 



In retrospect, the first three questions are sequentially related to outcome indicators of the 
entry, process, and completion phases. Quality of the candidate pool has been examined at the 
entry stage from past high school records (Question 1), and student motivation is monitored 
during the REVS-UP process to summarize quantitative and qualitative feedback from multiple 
sources (Question 2). In order to assess the long-term impact beyond the program completion, 
transcript studies are conducted below to evaluate the outcomes of past participants on college 
preparation (Question 3). 


Q3. How successful was REVS-UP on college preparation? 

Extensive efforts have been made to track college attendance information of past REVS- 
UP participants. The program coordinator kept a complete record of past high school students 
since 2010, and sent out e-mails to those participants eligible for college admission. 
Unfortunately, 34 e-mail addresses were identified as invalid. For the period prior to 2010, 
phone calls were conducted by the evaluator using participant information from application 


[7] http://novel.jschina.com.cn/yingyuwenxue/yinghmy/yinghaaaanmingyanl5.htm, p. 1. 
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forms. Because most phone numbers were outdated, a decision was made to sort past 
participants by schools. The evaluator e-mailed high school principals to solicit information of 
these former graduates. Nonetheless, not all schools kept the information on the college 
admission of its alumni, and most schools did not know whether REVS-UP participants have 
entered a STEM field. Therefore, online searches were subsequently conducted through various 
channels, including e-mail, Facebook, and Linkedln. As a result, 179 past participants were 
confirmed with college entry. Figure 7 shows the partition of their enrollment status across 
private and in-state schools (labeled “private”), out-state schools (labeled “OutState”), CSUB, 
other CSU campuses, community colleges (CC), and UC campuses. 

Figure 7: Enrollment of Accessible REVS-UP Participants in Higher Education 


CC 

20 


UC 

82 

According to the California Postsecondary Education Commission, 3.5% of Kem County 
high school graduates in 2007 attended a University of California (UC) campus, while the 
statewide figure was 7.3% [81 . Despite the traditionally low rate of UC acceptance in Kern 
County, REVS-UP participants attending UCs makes up the largest group of college-bound 
alumni (Figure 7). Omitting potential UC attendees from the unreachable portion of past 
participants, 82 students in Figure 7 already account for 21% of all 384 participants over past 



[8] see 2012 annual REVS-UP report, p. ix. 
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six years, including those from recent years yet to complete high school. The conservative UC- 
admission rate for REVS-UP participants is three times of the state average rate for high school 
graduates, and over six times of the corresponding rate for Kem County. 

Figure 7 shows CSUB as an attracting university for the second largest group of past 
participants (N=32). To compare student academic preparation between CSUB and other 
universities, high school GPAs have been averaged annually in Figure 8. The trend data indicate 
that CSUB admitted REVS-UP students with better high school GPAs only in 2008 (see blue 
line in Figure 8). When the results are aggregated across six years, the average high school GPA 
was 3.60 for CSUB students (see dotted blue line), and 3.83 for other university/college students 
(see dotted red line). The gap was statistically significant at a=.05 from an independent sample t 
test. 


Figure 8: High School GPA Comparison Between CSUB and Other Universities 

GPA 



To extend the result comparison to tertiary education, college GPA information has been 
extracted from transcripts of past REVS-UP participants at CSUB. A single sample t test was 
conducted to examine whether those students performed significantly above or below the general 
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CSUB student populations in the same period. As shown in Figure 9, REVS-UP students 
demonstrated significantly better performance than their peers of all undergraduate students at 
CSUB [t(31)=2.45, p<.05]. 

Figure 9: Comparison of Transcript GPA at CSUB 
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REVS-UP Participant at CSUB All CSUB Undergraduates 

Although CSUB did not always attract REVS-UP participants with high GPAs in high 
school (Figure 8), the REVS-UP group still outperformed their counterparts without REVS-UP 
experiences at CSUB. Since a strong link has been found between high school GPA and college 
performance (Stumpf & Stanley, 2002), the conclusion on solid college preparation is likely to 
be generalizable to other universities that admitted REVS-UP participants with better GPA 
records. For instance, one parent attested that “My son was in the [REVS-UP] program. He is at 
Cal Poly SLO as a biochemistry major, and doing very well” (personal communication on 
4/24/2013). 

In combination with the multilevel findings, REVS-UP has raised the quality of college 
preparation according to the GPA comparisons from secondary and tertiary education. 
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Meanwhile, REVS-UP participants showed a much higher rate of UC admission. Hence, the 
positive impact of REVS-UP is extended to both quality and quantity aspects of college 
preparation. 

Q4. In what aspects did REVS-UP strengthen professional development of K-12 

teachers? 

The shortage of qualified STEM teachers has been identified as the number 1 problem of 
U.S. education in a historical report entitled “A Nation at Risk” 191 . Perhaps because STEM 
knowledge at K-12 levels is relatively simple, reforms of teacher education often place more 
emphasis on improving teacher attitude, rather than subject competency (e.g., McDevitt, 
Heikkinen, Alcorn, Ambrosio, & Gardner, 1993; Sweeting, 2011). As a consequence, when the 
quality of STEM education was evaluated across countries in the 1990s, “Several Chinese 
scholars witnessed American teachers and students carrying out the wrong scientific experiments 
and calculations with great enthusiasm” (Su, Su, & Goldstein, 1994, p. 260). 

After entering the 21 st century, the issue of teacher education remained widespread in 
STEM fields. According to the California Council on Science and Technology 1101 , 10% of all 
secondary school teachers lacked appropriate qualifications as of the 2002-2003 school year. 

The misalignment between teaching and training is a phenomenon that has been confirmed in 
REVS-UP. Figure 10 is based on interview findings from a sample of 19 teacher participants, 
and the results show four of the STEM teachers possessing no bachelor degree in their respective 
subject fields. Thus, subject competency remains a critical area of teacher improvement in Kern 
County. 

[9] see http://mathcurriculumcenter.org/PDFS/CCM/summaries/NationAtRisk.pdf, p. 2. 

[10] see http://www.csub.edu/csubnews/2007/winter/chevron.shtml 
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Figure 10: Fields of Bachelor Degree Among Teacher Respondents 



To address the local need, REVS-UP demonstrated its effectiveness in engaging local 
STEM teachers in professional development. One teacher reported, “I enjoyed REVS-UP. I 
gained new knowledge in my subject area and improved my lab skills, invigorating my teaching 
and getting advice and help with lab equipment.” Another teacher echoed, “I think education is 
an ongoing process. Teachers need to seek further education in their care subject. Greater 
knowledge = greater teaching skills.” Improvement of teacher preparation is not confined to 
retention of existing knowledge, but also includes generation of new research outcomes. One 
teacher recollected, 

We were able to publish REVS-UP findings in two scientific journals and make five 
poster presentations at regional and national conferences. We participated in cutting- 
edge STEM research that has already taken place at CSUB. We also helped the 
development of experiments and lesson plans for K-12 STEM teaching. 

In addition to the strengthening of subject competency, REVS-UP has extended its 

impact on the development of positive attitudes among K-12 teachers. One teacher 

acknowledged that “REVS-UP has motivated me to teach to a greater depth and do more hands- 

on work.” Another teacher indicated that “REVS-UP has renewed my interest in science 

investigation, not just teaching.” The improvement of teacher performance has become so 
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evident at the program level, and one high school student projected that “I anticipate participating 
in the REVS-UP program in the future to become a better teacher.” 

While attitude and achievement are dual aspects of the program outcomes from K-12 
teachers, Stufflebeam and Shinkfield (2007) cautioned that "Too often, summative evaluation is 
carried out only forjudging programs or personnel. This restricts development processes and 
may lead to inadequate or even incorrect conclusions" (p. 24). To track the development 
process, teacher vitae is reviewed from REVS-UP application files to identify the baseline 
conditions. Directions of professional development are classified on the STEM discipline lines, 
and have been represented by red diamond symbols in Figure 11. In addition, color codes have 
been incorporated to differentiate education status of teachers at Bachelor (blue), Master (green), 
and doctoral (brown) levels. The STEM learning experiences are depicted in a network plot 
using NetDraw, a computer software package for social network analysis (SNA) (Borgatti, 

2002 ). 

Figure 11: Program Match Between Teacher’s STEM Training and REVS-UP Participation 
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Figure 1 shows variation of the network density across subject domains. While 
mathematics is a separate subject, science is divided into biology, chemistry, geology, and 
physics. Although no teacher indicated concurrent participation in both biology and physics 
projects, a couple of teachers took part in STEM projects across multiple disciplines (see the 
circled area). Biology, mathematics, and physics projects have also attracted teachers from non- 
STEM majors. Among the 19 teacher participants, 11 gained interdisciplinary learning 
experiences through REVS-UP. The remaining teachers pursued knowledge enrichment in their 
initial fields of STEM education. Despite the relatively small sample size, Figure 1 1 
demonstrates extensive learning opportunities for K-12 teachers to strengthen their STEM 
knowledge in REVS-UP. 

California State University (CSU) has been the largest producer of mathematics and 
science teachers in the state. At the inception of REVS-UP, it was projected that the demand for 
new STEM teachers in California would exceed 22,000 during 2007-2012 1111 . Guided by the 
commitment of the CSU system to teacher preparation, REVS-UP has illustrated its sustainable 
mechanism to enhance teacher education on multiple aspects, including knowledge acquisition, 
research dissemination, and attitude improvement. These findings have been triangulated by 
quantitative and qualitative data from teacher interviews, questionnaire responses, vitae analyses, 
project profiling, and SNA plotting. Because teacher quality is one of the most important 
determinants of student achievement, feedback from high school students has also been cited in 
this section to confirm the benefits of professional development for K-12 teachers in REVS-UP. 

Q5. How effective did CSUB student assistants perform as REVS-UP partners? 

David Hamburg, President of the Carnegie Corporation of America, pointed out, “Every 


[1 1] http://www.calstate.edu/newsline/2007/n20070223bakl.shtml 
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college and university in the nation should have a strong, substantive, explicit, functional linkage 
with schools in its geographical area” [121 . REVS-UP has done this by establishing a sustainable 
platform to connect CSUB and local schools. In addition to enriching learning opportunities for 
high school students and K-12 teachers, the program involved CSUB student assistants to 
facilitate STEM project development. According Campus Compact (2013), “College students 
make excellent mentors [for high school students] because they are close enough in age to young 
people to establish strong relationships, yet mature enough to offer guidance” (][. 10). 

The effect of role modeling is demonstrated by its support in strengthening the 
motivation of high school students in REVS-UP. Because “The spectator sees more of the 
game”, survey responses have been gathered from 19 participating teachers who witnessed the 
team member interactions. When CSUB student assistants were present as role models, 92% of 
the K-12 teachers observed positive motivation among high school students in REVS-UP. A 
Mantel-Haenszel Chi-Square test was conducted to reconfirm the significant association in 
Figure 12 [ x 2 (l)=7.00, p=0.0083]. 

Figure 12: Association Between Motivation and Role Modeling 
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During an interview, one teacher reported, “I think the most rewarding experience was 
working with motivated and inquisitive young men and women and seeing the level of high 


[12] http://www.compact.Org/resources/mentoring/3697/#why-campus-based 
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quality work they were able to produce.” A contingency table analysis shows a substantial 
association between the performance of CSUB students and the knowledge gained by K-12 
teachers (C=.62), and the linkage is statistically significant at a=.005 [x 2 (2)=ll .25, 
p=.0036]. During an interview session, teachers acknowledged the contribution of CSUB 
student assistants to the learning process. One teacher indicated that “Working with the 
professors and student assistants is my favorite part of the program. I learn a lot of new 
material.” Another teacher recollected that “I enjoyed interacting with the high school and 
CSUB students the most. I’m glad this program exists!” 

Besides their positive impact on high school students and K-12 teachers, CSUB 
students also assisted in leading STEM investigations. In a survey questionnaire, teachers 
were asked to rate the performance of both CSUB professors and student assistants, and a 
significant association was found between those two variables [x 2 (4)=18.24, p=.0011]. The 
effect of this teamwork has led to rejuvenation of teacher interest. One teacher attested that “I 
also was able to work with a professor and student teaching assistants. It was interesting.” 
Except for one missing response, all K-12 teachers rated CSUB students as good or very good 
partners (Figure 13). 

Figure 13: Partnership Rating of CSUB Student Assistants 



According to the Community Service Act of 1990, service learning is defined as a 


framework “under which students or participants learn and develop through active participation 
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in thoughtfully organized service” ll3J . CSUB student assistants filled this pivotal role in REVS- 
UP to support team-based, collaborative inquiries in the STEM fields. As Andreas Gebauer, the 
REVS-UP Director, pointed out, “This is a one-of-kind program that offers our community 
the opportunity to participate in research projects that happen every day at CSUB” [14] . 
Because of their familiarity with the CSUB learning environment, student assistants are 
able to effectively engage and strengthen teamwork through their sendee -learning 
support for high school students, K-12 teachers, and CSUB professors. 

Q6. How well did professors demonstrate academic leadership in STEM research? 

In general, the performance of CSUB professors has been evaluated regularly in three areas, 
teaching , scholarly activity , and sendee, according to the university faculty handbook 1151 . REVS-UP 
provides a professional opportunity for faculty members to illustrate their balanced performance in 
all three areas. As one teacher noted, 

The faculty enjoy what they are doing; The faculty are invested in their students’ learning 
[area of teaching]; The faculty continues to learn new information & utilize in the teaching 
and research [area of scholarly activity]; the faculty’s research is relevant and important to 
what is going on in the world [area of service] . 

The STEM expertise of the professors has won them the respect of REVS-UP participants. 
One teacher reported that “They are very knowledgeable and extremely professional! The CSUB 
science professors I’ve worked with are great educators and scientists.” As shown in the vitae files, 
all participating professors earned a doctoral degree in one of the STEM fields. Nearly all research 
REVS-UP projects were developed in the original field of their Ph.D. training. One exception was 
related to a physics professor, Dr. T., who switched his research interest to satellite image tracking 

[13] http://eodes.lp.findlaw.eom/useode/42/129/I/A/125 1 1 

[ 1 4] http://people.bakersfield.eom/home/ViewPost/l 2 1278 

[15] http://www.csub.edu/facultyAffairs/files/handbook/UniversityHandbook.pdf 

of environmental changes. The environmental research had made him a leading scholar 
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of environmental changes. The environmental research had made him a leading scholar in valley 

fever studies, and by the REVS-UP inception, he already had three articles published in 

nationally refereed journals of this field 1161 . Furthermore, the self-learning experience has 

enhanced his teaching style. One REVS-UP participant recalled, 

The CSUB professor that I worked with was Dr. T. and he’s a good teacher. He explains 
things briefly and clearly without droning on at length. He is also enthusiastic about his 
work which creates enthusiasm in his students. He is patient and happy to repeat things 
as needed without getting upset. I found his style to be effective with the high school and 
grad students as well as the teachers in his classroom! He was punctual and prompt. Dr. 
T. . . is a great role model. 

The commitment to STEM education is demonstrated among other professors as well. 
One teacher concluded, “The project mentors were very supportive of the teachers and students. 
They did an awesome job! I really enjoyed working with them.” Another teacher concurred that 
“The project mentor worked very closely with us and was great.” Across all the teacher respondents, 
the survey findings showed a significant association between professor rating and K-12 teacher 
learning [% 2 (2)=8.51, p=.0142|. The professors’ mentorship also benefited high school students, 
which led one teacher to write “I wish I had REVS-UP opportunity when I was in high school.” 
The quality of the STEM projects is guaranteed by the academic leadership of the CSUB 
professors. One teacher reflected, “Being in the research itself is a professionally benefit because 
you’re working with experts in the field.” Over the past six years, a total of 26 professors took part 
in REVS-UP, and 1 13 research presentations have been made and disseminated online 1171 . Due 
credit has been given to the team leaders, and a participant reported, “Project mentor was very 
helpful; he was serious about the research but readily joked around w/ the participants; gave clear 
good or very good partners. 

[16] http://www.csub.edu/~jtalamantes/vitae.pdf 

[17] http://www.csub.edu/stem/ 

and concise directions.” Figure 14 shows that nearly all teacher respondents have rated professors as 
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Reciprocally, more than 84% of the teacher respondents believed that STEM 
projects have helped professors advance their research agenda. This outcome addresses 
one of the original purposes of the CSUB-Chevron partnership, i.e., to “help faculty with their 
research” 1181 . As a result, REVS-UP has increased its attractiveness to CSUB professors, and the 
trend of participation shows the highest number of professor participants in 2012 (Figure 15). 


Figure 15: Trend of Professor Participation During 2007-2012 



Apparently, the 2009 result is an outlier in Figure 15. This was because REVS-UP was 
anticipated to end in 2009 according to the initial agreement between Chevron and CSUB 1 19] . 
Sustaining REVS-UP was grounded on a mutual vision of the private-public partnership to 
support STEM education. As Horace Mitchell, CSUB President, pointed out, “We 


[18] http://www.bakersfieldcalifomian.corn/local/x504580726/Chevron-donates-500-000-for- 
summer-research-program 

[19] http://www.calstate.edu/newsline/2007/n20070223bakl.shtml 
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want to get students in the middle schools, junior highs and high schools excited about studying 
science and math. We also want to work with teachers to better prepare them to teach these 
subjects and enhance their knowledge.” In this broad context, CSUB professors have designed 
STEM projects to address these needs, and thus, become a core asset of the partnership building. 

Since REVS-UP involves high school students, K-12 teachers, CSUB student assistants, 
and university faculty, evaluation findings have been analyzed in this report to highlight both the 
teaching and learning aspects of STEM experiences. The teaching part primarily worked for 
professors and their assistants, but the learning part was not solely confined within high school 
students and/or K-12 teachers. Professors received support from REVS-UP participants to help 
complete their STEM research projects, and CSUB student assistants gained service-learning 
experiences. The joint effort has established a community of learners for all stakeholders at 
CSUB. As one K-12 teacher characterized it, ‘‘REVS-UP has made the CSUB research 
environment more professional and yet creative for all of us at the same time.” 

Q7. In what areas did REVS-UP show continuous improvement over past six 

years? 

In this report, a question-oriented approach has been taken to streamline multilateral 
evaluation findings. While the accountability of REVS-UP has been addressed in Questions 1-6 
for its target populations of high school students (Q1-Q3), K-12 teachers (Q4), CSUB student 
assistants (Q5), and STEM professors (Q6), both formative and summative evaluations are 
needed to assess the impact of continuous improvement (Stufflebeam & Shinkfield, 2007). As 
Sloane (2008) suggested, “We change the basic research question from what works to what 
works for whom and in what contexts” (p. 43). Guided by the Context, Input, Process , and 
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Product (CIPP) paradigm (see Stufflebeam, Madaus, & Kellaghn, 2000), this section is devoted 
to the examination of ongoing progresses on three benchmarks: 

(1) Addressing “what works” through evaluation of quality improvement in the STEM 
project development; 

(2) Investigating the “for whom” component to delineate service coverage for diversified 
high school student populations; 

(3) Describing contextual changes to reflect ongoing enhancement of program 
management. 

The product phase of the CIPP model is examined in the next section to assess REVS-UP impact 
in K-12 settings. 

Improvement on STEM Project Development 

STEM projects from the past have been posted online at http://www.csub.edu/STEM. 
Built on the annual outcomes, this report places more emphasis on the trend of continuous 
improvement over past six years. In comparison to other years, 2009 was the final year of the 
first funding cycle (2007-2009). Due to the resources dwindling down, REVS-UP managed to 
leverage additional support from other sources, including an NSF-sponsored project entitled 
“Vascular Transport of California Native Shrubs”. 

Figure 15 shows the total number of presentations over past six years. Except for 2009, 
REVS-UP has demonstrated a trend of expanding STEM project developments (Figure 15). The 
project count was based on the number of annual research presentations at the end of each 
summer. As a comparison, the number of STEM projects increased from 18 in 2007 to 23 in 
2012, a 28% increase during the six-year period. This program growth was in response to local 
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needs - when K-12 teachers were asked about the changes they would like to see in this 
program, several of them indicated the need for expansion. 

Figure 15: Number of STEM Presentations Developed from REVS-UP Each Year 
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Besides adding projects to increase STEM learning opportunities, more progress has been 
made to broaden the subject domains of REVS-UP inquiries. For instance, the orange line in 
Figure 16 shows no computer science projects in 2007. To compensate, four new projects were 
added to the subject in 2008. The increase of project variety has made REVS-UP more attractive 
to local stakeholders. As one K-12 teacher suggested, “More variety in projects would be 
beneficial. Many students enjoy Robotics, as my son did last summer. How about Robotics I 
and II where students could continue more advanced projects in the second level?” 


Figure 16: Trend on Project Numbers Across Subject Domains 
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While the trend of research developments has been monitored at both program and 
subject levels (Figures 15 & 16), indicators of the project quality are tracked by the ratings of 
STEM presentations each year. Hence, continuous improvement can be illustrated by a 
longitudinal data comparison across time. For example, a project entitled “From the Periodic 
Table to the Kitchen” won the 3 rd place award in 2008. In 2012, the same project was rated at 
the 1 1 th place. The comparative result shows that nearly half of the 2012 projects could have 
been rated within the top 3 ranks in 2008. 

Figure 17 shows names of the professors who led the development of STEM projects for 
the top-3 awards during 2007-2012 [20] . The continuous improvement has been further illustrated 
by consistency of the project evaluations in honoring outstanding STEM professors for multiple 
years (see Professors Barton, Horton and LaFever in the blue-colored and orange-colored frames 
in Figure 17). 

Winning the REVS -UP award was no easy task. According to the trend results, Professor 
Kemnitz’s project was rated at the first place in 2007. In 2008, two other projects surpassed his 
new project during the process of continuous improvement, and moved his project rank to the 
third place (see the yellow-colored frame in Figure 17). The stiff project competition in 2008 
also led to a decision of sharing the third place award by two projects in chemistry. 

In summary, the longitudinal data have provided significant insights into the continuous 
improvements of the STEM projects over the past six years. Ongoing progresses have been 
made to increase the project numbers, broaden the subject coverage, and enhance the quality of 
STEM discoveries through peer-reviewed research competitions. 

[20] The award competition is based on Research Project Competition since 2007. Results from 
Exploring Science and Mathematics Competition will be addressed in the next section. Because 
of dwindling of the funding, no awards were given in 2009 in either competition. 
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Figure 17: Network of Awardees and Award Ranks 



Services for Diversified Student Populations 

A core feature of REVS -UP is the concurrent enrollment of high school students in a 

college setting. As Ulate (2011) recapped, 

Concurrent enrollment (CE), defined as a high school student enrolling in college 
coursework, provides high school students with a unique opportunity to enhance their 
academic experiences. Originally, CE was solely intended to provide high-achieving 
high school students with opportunities to enrich their education by enrolling in college 
courses. Recently, CE also has been seen as a promising strategy to improve the college 
readiness levels of underrepresented students, and to reduce disparities in college access. 
(P- 1) 

Through sponsoring STEM investigations, REVS-UP has broadened its impact on both 
the quality and the equity of college preparation for diversified student populations. Past REVS- 
UP participants came from 49 different high schools with an average school rating at the 56.2th 
percentile [21] . Since two of the high schools (Centennial and Stockdale) received 9 and 8 ratings, 
most other schools were ranked below the median level. 


[21] The scale was developed by greatschools.com according to student performance on state- 
required tests. 


REVS-UP kept records of the minimum high school GPA for its participants during 
2007-2012 (Figure 18). The results confirmed that REVS-UP did not exclude high school 
students with weak academic background. When REVS-UP started in 2007, George W. Bush’ 
administration pushed for “No Child Left Behind” (NCLB), and thus, the trend in Figure 18 
shows the program alignment with the original intention of the NCLB initiative to close equity 
gaps in college preparation. 

Figure 18: Trend of Minimum High School GPA for REVS-UP Participants 



According to California Postsecondary Education Commission (2007), “California has 
serious inequities in access to higher education. College-going rates vary greatly depending on 
students’ ethnicity, gender, and the type of neighborhood where the student’s high school is 
located” (p. 1). Kem County has been ranked as one of the lowest regions in adult education 
across the United States (Brookings Institution, 2010). Zumbrun (2008) concurred that 
Bakersfield, the county seat, was ranked as one of the least educated metropolitan areas across 
the nation. The 2010 Census data indicated 49.2% of Kem County residents come from a 
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Hispanic background 1221 . The state index suggested a college-going rate of 38.7% in the Hispanic 
population 1231 . In combining both indicators, it is projected that students of Hispanic background 
only makes up about 19.0% of the college-bound population. Meanwhile, the efforts of data 
tracking have revealed college enrollments of 179 REVS-UP participants, and the proportion of 
Hispanic students has reached to 29.4%, much higher than the projected 19.0% for Kem County 
(Figure 19). 

Figure 19: Ethnic Distribution of College Bound Participants 
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On the gender dimension, College Board further indicated that “Male students are about 
twice as likely as female students to enter STEM fields” 1241 . Except for 2009, the trend data in 
Figure 20 showed an increase of female student participation in REVS- UP, reversing the general 
pattern of gender inequality that persists across the nation. 

Figure 20: Trend of Percent Female Participation During 2008-2012 
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[22] "2010 Census P.L. 94-171 Summary File Data" (www.census.gov/2010census/data). 

[23] http://www.cpec.ca.gov/StudentData/CACGREthnicity.asp 

[24] http://trends.collegeboard.org/education-pays/figures-tables/students-stem-fields-gender- 
and-race-ethnicity 
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Broadening the population coverage directly supports the dual role of STEM education to 
prepare STEM majors and general citizens concurrently. REVS-UP has displayed its services to 
the diversified populations of high school students across dimensions of race, gender, and 
academic preparations. 

Enhancement of REVS-UP Management 

Contextual changes occurred in past six years to enhance management of REVS-UP on 
both aspects of capacity building and project support. As the program embarked on a new 
expansion in 2010, a program coordinator was hired to address the service needs of REVS-UP 
participants based on feedback from the first funding cycle during 2007-2009. In particular, one 
student reported in 2008 that “Some areas of improvement needed but good organization overall; 
some students didn’t receive the letter of information detailing the time, date & place of the first 
meeting; info on free parking should be sent out before the program begins.” 1251 The 
organizational refinement has been maintained since hiring the coordinator, and no further 
complaints have been reported against the administrative support. 

While capacity building has solved issues from the past, continuous improvement 
becomes an ongoing process to support project enhancement during the REVS-UP operation. In 
2008, REVS-UP made an important improvement over 2007 by splitting STEM projects into two 
tracks: (1) Direct Research Participation (DRP), and (2) Designed-to-Learn Development 
Research Participation (DLDRP). DRP projects included cutting-edge research in STEM fields, 
while DLDRP projects placed more emphasis on the development of experiments and lesson 
plans for STEM education. Parallel to the original Research Project Competition , REVS-UP 
added an Exploring Science and Mathematics Competition since 2010 to recognize exceptional 


[25] The 2008 annual report of REVS-UP 
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projects in the DLDRP group 1261 . As of 2012 [27] , DRP projects have resulted in three publications 
in scientific journals and five poster presentations at regional and national conferences. 
Furthermore, the DLDRP project in chemistry produced manuals with experiments suitable for 
K-12 schools. Thus, the inclusion of DRP and DLDRP tracks has strengthened REVS-UP 
alignment with the dual emphases on STEM education, and improved STEM learning 
opportunities for both prospective STEM majors and in-service K-12 educators. 

Continuous improvement also occurred with assessment tools used to support project 
evaluation. Initially, REVS-UP adapted an existing rubric from a Student Research Scholarship 
(SRS) program at CSUB. Similar to REVS-UP, SRS supported development of poster 
presentations. However, the SRS funding from the President’s Associates was not exclusively 
designed for STEM education, nor did it capture the multilateral collaboration among high 
school students, K-12 teachers, CSUB student assistants, and STEM professors. 

In 2010, the REVS-UP director re-examined the program needs, and developed a new rubric 
after an extensive review of the current literature, including a widely-used rubric entitled “Valid 
Assessment of Learning in Undergraduate Education” (VALUE). As a result, the new rubric 
includes six dimensions, Context of and Purpose for the Poster, Content Development, 
References and Evidence, Composition and Format, Conclusions and Related Outcomes, and 
On-site Explanation. On each dimension, a project is rated at four levels, Emerging, Basic, 
Proficient, and Exemplary. Indicators of project attainment are clearly defined in a 6x4 matrix to 
enhance feasibility of the rubric application. The new rubric has since been adopted for the 2011 
and 2012 STEM project assessments. Meanwhile, it also provides a model to lead other 


[26] The 2010 annual report of REVS-UP 

[27] The 2012 annual report of REVS-UP 
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assessment tasks, including the SRS competition. 

In conclusion, REVS-UP has considered program needs and service feedback to make 
continuous improvements on several fronts, such as accommodating more STEM projects and 
supporting new explorations in computer science. The hiring of a REVS-UP coordinator has 
also strengthened the office capacity to serve diversified populations of high school student, K- 
12 teachers, CSUB student assistants, and STEM professors with various responsibilities and 
expectations. As the program incorporates more STEM projects to enrich the learning 
experiences, a new rubric has been developed to assess quality of research outcomes of both 
DRP and DLDRP projects on six dimensions. Meanwhile, REVS-UP director has established 
professional leadership via ongoing improvements of the project tracking and poster evaluation. 

Q8. What progress has been made to sustain REVS-UP impact in K-12 settings? 

Through STEM project developments, collaborative partnerships have been established 
among REVS-UP participants to sustain the impact in K-12 school settings. Based on the 
involvement of high school students and K-12 teachers, student learning, teacher preparation, 
and network building are examined in this section to assess the long-lasting outcomes resulting 
from REVS-UP. 

Inquiry-based Learning 

REVS-UP has adopted effective strategies to enhance STEM education. While 
supporting inquiry-based learning, REVS-UP incorporates STEM explorations beyond the level 
of secondary education. As one teacher reported, “I had the chance to work not only on my own 
but with chemicals and machines that were never available in my high school classes.” Another 
teacher concurred, “I was able to work in an actual lab, unlike the ones at school. I learned much 


more than I thought I would.” 
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This effective approach is aligned with the professional practice of program development. 
According to Lev Vygotsky (1978), a seminal educator, learning activities should be designed 
within “the zone of proximal development” (ZPD), and thus, the content knowledge needs to be 
positioned slightly above the level of student development. The concurrent enrollment of high 
school students in the program thus fits the goal of a collaborative exploration beyond secondary 
education. 

Based on Vygotsky’s (1978) original definition, ZPD represents the distance between the 
actual development level of the learners and the level of potential development in collaboration 
with more capable peers and experts. Therefore, teamwork is an essential component to support 
the learning process. REVS -UP provided opportunities for collaboration among professors, 
CSUB student assistants, K-12 teachers, and high school students. This teamwork was evident 
in the program, with one participant noting that “Our group of high school students performed 
various experiments and techniques to get our results. The professor and CSUB student were very 
knowledgeable. We all learned to work together in our project.” 

“As a learner gains new skills and abilities, this [ZPD] zone moves progressively 
forward” (Cherry, 2013, ( J[. 3). To continue the ongoing progress, it is important to recognize 
ZPD as a moving target. Thus, STEM explorations in REVS-UP facilitate the advancement of 
student knowledge base. As one student acknowledged, “I was perplexed by several advanced 
STEM topics during my school years. I signed up for a project in quantum chemistry, and 
learned a lot when I was in the REVS-UP program.” Meanwhile, flexibility has been included to 
foster creativity in the collaborative inquiries. Another student welcomed the flexibility aspect - 
“REVS-UP is very well organized and productive. I appreciate the fact that there is a lot of 
freedom to work at different paces.” 
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In summary, REVS-UP has incorporated new inquiries beyond secondary education to 
enrich the learning process for high school students and K-12 teachers. The inquiry -centered 
pedagogy, ZPD-based curriculum setting, and creativity-oriented flexibility have supported a 
proper alignment between the REVS-UP approach and the professional practice advocated by 
Lev Vygotsky, a leading educator of the 20 th century. 

STEM Teaching 

In addition to enhancing subject competency through STEM research inquiries, REVS- 
UP supports the development of instructional tools and lab techniques to assist K-12 teachers in 
classroom settings. Although the state government has placed more emphasis on paper- and- 
pencil tests, “It remains true that students are attracted more to the STEM fields when 
applications and hand-on experiences are an integral part of the curriculum'’ (Meyer & 
Gasparayan, 2010, p. 77). 

With the support of local teachers, instructional tools developed from REVS-UP have 
impacted STEM learning across multiple disciplines. For instance, lab manuals have been 
developed from a project entitled “Fun with Science: Bringing Hands-On Experiments to 
California Classrooms.” This tool was adopted by STEM teachers to enhance the learning 
experience of about 1,000 students at local schools. Another project, From the Periodic Table to 
the Kitchen: Cooking with Circuits, has produced five sets of lab instructions that have impacted 
STEM education for at least 5,000 students in Kem County. These tools are user-friendly, and 
provide practical templates for electrochemical experiments. Furthermore, the effectiveness of 
the instruction tools has been disseminated at regional and national conferences of the American 
Chemical Society [28] . Similar projects have also been developed in physics labs during the 2010 


[28] see 2012 Chevron Report 
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REVS-UP sessions (Meyer & Gasparayan, 2010). 

The impact of REVS-UP was sustained when STEM teachers implemented a plan to 

incorporate the inquiry -based learning into local schools’ curricula. One of them recollected, 

REVS-UP gave me insights into inquiry-based learning. I modified my teaching style to 
give students more opportunities to discover the answers to questions through their own 
investigations. I also created more labs that allowed students to define and record their 
own procedures for collecting data to answer a scientific question. This was difficult in 
some instances because the students are accustomed to just being told what to do rather 
than guided to the correct procedure or answer. However, I think it ultimately made my 
classes more engaging to students in the same way as REVS-UP, but on a smaller scale. 

As a result of these efforts, survey responses from 19 teachers revealed different levels of 

teaching improvement. As shown in Figure 21, the majority of teachers indicated a great deal of 

teaching improvements. 

Figure 21: Proportion of Teachers with Improvement of Teaching Performance 


Meanwhile, the impact of REVS-UP was not solely confined to the K-12 teacher 
participants. Based on responses from a survey questionnaire, around half of the teachers gained 
more respect from their peers who did take part in REVS-UP. The increase of peer respect was 
significantly associated with the extent of teaching improvement [% 2 (2) = 8.97, p = .0113]. As 
shown in Figure 22, all teacher participants gained peer respect when a great deal of teaching 
improvement occurred in local schools (the top red circle). 
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Figure 22: Association Between Peer Respect and Teaching Improvement 



The enhancement of STEM teaching is much needed in California. As CSUB President 
Mitchell (2007) pointed out, “While the state has adopted a set of ambitious academic standards, 
and worked to expand and improve teacher production, the shortage of qualified teachers has 
actually risen in science and math” 1291 . REVS-UP has addressed this challenge by facilitating 
teacher growth. A new teacher indicated, “As I will be teaching high school chemistry for the 
first time in my career, my REVS-UP participation helps me tremendously in preparing new 
lessons at my school.” 

Over the past six years, REVS-UP has sustained its impact on STEM teaching through 
the development of instruction tools and lab techniques. As teaching performance has improved 
in local schools, the useful lab manuals created from REVS-UP are disseminated in national and 
regional conferences of professional organizations, and peer respect has been established for K- 
12 teachers who demonstrated a great deal of teaching improvement. 

Network Building 

The impact on K-12 schools is not confined to improved teaching methods. REVS-UP 
also provides an important opportunity to facilitate network building among educators and 
students in the community. One teacher noted, “I enjoyed REVS-UP because it offered a chance 
to work with others. Teaching high school physics can be awfully lonely sometimes.” To 


[29] http ://ww w .calstate.edu/newsline/2007/n2007 0223bak 1 . shtml 
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analyze the progress on network building, survey questionnaires have been distributed to teacher 
participants to identify beneficial partners during the STEM project exploration. Based on the 
responses from 20 teacher participants, the social network analysis revealed an average of three 
beneficial partners per teacher (Figure 23). In particular, 28% of the networks are connected to 
teacher partners, and 27% of the linkages are tied to professors. The networks with high school 
students and CSUB student assistants account for 24% and 21% of the partner relations, 
respectively. Therefore, no particular group of partners has predominantly controlled the process 
of partnership building. 


Figure 23: Beneficial Partnerships Perceived by K-12 Teacher Participants 



Although the network pattern seemed to suggest more partnership building among 
teachers (28%) and between teachers and professors (27%), the importance of student 
partnerships has also been recognized in interview findings. For instance, one teacher indicated 
that “Student input in labs is helpful and necessary. I value student opinions and have a greater 
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understanding of what type of curriculum benefits them.” Another teacher reported, “I was able to 
work with a Cal State student who was very knowledgeable about organic chemistry and taught me a 
lot.” 

Since the whole can be larger than the sum of its parts, REVS-UP not only attached great 
importance to the quality of STEM learning at the individual level, but also improved network 
building as a group outcome. Through partnership collaborations, new STEM inquiries have 
been completed that enrich teaching to align with “the zone of proximal development” (ZPD), 
and instructional tools have been created to expand the inquiry-based learning in local schools. 
Thus, the impact on K-12 education is reflected by both the teaching and learning aspects of 
REVS-UP support. 

Q9. What recommendations can be made to extend success of REVS-UP beyond 

this summative evaluation? 

Since 2007, Chevron has funded REVS-UP at CSUB in order to offer hands-on research 
experiences for high school students and K-12 teachers. Over the past six years, a total of 26 
STEM professors led the development of 1 13 research projects with the support of 90 CSUB 
student assistants, 384 high school students, and 84 teachers from K-12 schools. In this report, 
qualitative and quantitative data have been analyzed to examine the impact of REVS-UP on high 
school students (Questions 1-3), K-12 teachers (Question 4), CSUB student assistants (Question 
5), and STEM professors (Question 6). In addition, sustainable trends have been analyzed to 
summarize program improvement (Question 7) and capacity building (Question 8) under the 
Context , Input, Process, and Product (CIPP) paradigm. 

On the basis of the extensive examination of REVS-UP outcomes in Questions 1-8, this 
section is devoted to summarizing five recommendations to expand the impact of REVS-UP 
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beyond this longitudinal investigation: 

1. Continue the support for inquiry -based learning 

All STEM projects sponsored by REVS-UP have incorporated knowledge content 
beyond the level of secondary education. This setting is pertinent to the concurrent enrollment 
of high school students at the college level, and facilitates STEM learning within “the zone of 
proximal development” (ZPD). While consideration of ZPD is needed to optimize learning 
outcomes from a specific project, a systematic approach can be taken to extend the inquiry-based 
learning across multiple projects. For instance, one teacher suggested having Robotics I and II to 
match ZPD advancement of high school students at different levels. 

(2) Use the REVS-UP platform to leverage additional support 

REVS-UP is an example of a sustainable program model that improves the quality of 
STEM education in a traditionally underserved region of the United States. The private and 
public partnership between Chevron and CSUB has not only increased student interest in STEM 
explorations, but also has enhanced subject competency of K-12 teachers. Meanwhile, the U.S. 
government has developed several programs to revitalize STEM education for the next 
generation (President’s Council of Advisors on Science and Technology, 2012). In 2009, the 
REVS-UP platform was extended to sponsor additional STEM research funded by the NSF 
CAREER program. In 2012, another NSF grant award was given to add more research 
opportunities through information assurance education (ISE) [301 . While the existing REVS-UP 
program already covered the major STEM subjects, the ISE award addressed a specialty field of 
national security, and thus, created a new dimension for future REVS-UP expansion. 

(3) Strengthen the program alignment with dual emphases on STEM education 


[30] http://www.nsf.gov/awardsearch/showAward? AWD_ID= 1241636 
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Two separate emphases have been introduced in REVS-UP to classify STEM 

investigations on dual tracks: (1) Direct Research Participation (DRP) in cutting-edge research, 

and (2) Designed-to-Learn Development Research Participation (DLDRP) to develop 

experiments and lesson plans in the STEM fields. Although the DRP projects are less relevant in 

the K-12 school context, experiments and lesson plans from DLDRP could depend on the 

characteristics of the student populations in the local setting. For instance, lesson plans that 

worked for an Advanced Placement class may not be equally effective for a remedial class. The 

student engagement issue was observed by REVS-UP participants, with one teacher noting: 

I think students should be interviewed or checked more carefully. One of the student 
participants was difficult to work with and did not really meet the prerequisite skills 
required to participate. She was also not self-motivated, so it made it difficult to work 
with her in a group setting. The other students in the group were fantastic. 

Thus, DLDRP projects are now expected to indicate what works, for whom, and in which 

contexts. This additional tracking effort will enhance the alignment of REVS-UP with the dual 

emphases of STEM education for the general public and professional experts. 

(4) Facilitate strategic collaboration with other STEM programs 

CSUB has offered multiple programs in STEM education. In 2013, NSF alone has 

funded three projects totaling over $2 million (award # 0934944, 1136342, and 1241636). 

During the past six years, Chevron has also contributed money to support the following 

programs: 

1. Chevron High School Academy 

2. Chevron- sponsored STEM Student Center 

3. Chevron-sponsored dual credit geology program 


4. Chevron-sponsored development of a bachelor’s degree in engineering sciences 
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5 . The Chevron Math Initiative 

6. Chevron CSUB-NASA/JPL 2008 Summer Institute 

REVS-UP participants from local schools are allowed to concurrently participate in multiple 
programs. Thus, strategic collaborations with other STEM programs are recommended to help 
sharpen the unique role of REVS-UP under this broad context. 

(5) Strengthen program coordination to address participants’ needs 

In preparing this report, all of the past participants had a chance to provide their 
feedback. Therefore, some of the concerns from the early years could have already been 
resolved by REVS-UP. For that reason, four specific aspects are combined below with 
supporting evidence for the reference of REVS-UP staff: 

(i) Schedule Coordination - One participant suggested, “Bring back lunch. Even if it’s 
just sandwiches, it saves a lot of time and money for the students.” Another student 
reported, “I loved our schedule. We had morning lecture on advanced topics and 
background. The afternoons were left for experimentation”; 

(ii) Program Application - One teacher noted, “The only change, perhaps, is the digital 
application process. Having to hand- write so many recommendation letters for 
students (over 20 this year [2013]) gets quite time consuming”; 

(iii) Project Assignment - One past student urged the program to have, “the right numbers 
of students and teachers assigned to each project so that no team members are 
overwhelmed while others have nothing to do”; 

(iv) Time Commitment - Since college credits have been designated for REVS-UP, the 
time commitment of STEM professors should be aligned with similar course 
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offerings at the state side. Interview respondents have indicated the need to 
standardize the time commitment. 
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